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2 (57) Abstract: A method and system that optionally allows a user to view image defects organized by natural groupings based on 
^ features of the images. The natural groupings make it easier for the user to organize some or all of the images into classes in a training 
^ set of images. A feature vector is extracted from each image in the training set and stored, along with its user-specified class, for 

use by an automatic classifier software module. The automatic classifier uses the stored feature vectors and classes to automatically 
Q classify images not in the training set. If the automatically classified images do not match images manually classified by the user, 

the user modifies the training set until a better result is obtained from the automatic classifier. The system can provide feedback to 
^ an inspection system designed to aid in the setup and fine-tuning of the inspection system. 
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Currently existing optical inspection systems with automatic defect classification 
require human beings to visually inspect semicondutor wafers for suspected defects and to 
classify the types and causes of the defects in order to set up the automatic classification 
system. To perform this classification, a human being must sort through hundreds of 
5 images that are presented in random order. This process takes many hours and increases 
the cost of production. Moreover, even skilled human operators are somewhat slow and 
prone to error. 

Summary of the Invention 

10 The described embodiments of the present invention receive images of defects and 

aid a user in classifying the types of defects represented. A graphical user interface allows a 
human user to manually classify the defect images via Automatic Supervised Classifier 
software and further allows the user to contrast his manual classifications with the 
classifications determined by Automatic Supervised Classifier software. In order to create 

15 an Automatic Supervised Classifier for semiconductor defect classification, the human user 
has to perform two manual tasks: 

(i) Creation of a good Classification Scheme (which images the user will place into 
which classes). 

(ii) Creation of a good training set of examples for the Automatic Supervised 
20 Classifier with this Classification Scheme. 

The described embodiments of the invention provide a tool to help the human user 
achieve both these objectives in record time. 

The embodiments described herein contain four main components that seamlessly 
interact with one another: 
25 (i) Image Gallery: This is a graphical interface to display a list of images in an 

organized fashion. 

(ii) Dynamic Automatic Supervised Classifier: Given a list of defects, the user is 
allowed to manually classify or train any set of defects. The rest of the defects are 
dynamically classified accordingly and the overall performance of the resulting classifier is 

30 calculated. 

(iii) Dynamic Classifier Controls and Performance Tools: This allows the user to 
visualize and optimize the parameters of the classifier even further. As in (ii), the resulting 
performance is immediately visible. 
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Detailed Description of Preferred Embodiments 
The described embodiments of the present invention aid a user in classifying defect 
images input to the system from an outside source. For example, the images can be 
received from a scanning electron microscope (SEM), a defect inspection system, such as 
5 KLA-Tencor's 2132 inspection tool, or any similar instrument used to generate images and 
detect errors therein. In certain embodiments, the images can come from more than one 
type of input source. For example, images may be received from both an SEM and a 2132 
inspection tool. Different types of images received from different sources can aid in defect 
classification, since different types of images produce more information is provided 

10 concerning the defects to be classified. The images also can be point locations on wafers. 
Fig. 1(a) is a block diagram showing an overview of a semiconductor optical, 
ebeam, or other types of inspection systems. As discussed above, defect images are 
preferably received from an outside source, such as an optical, ebeam, or other types of 
inspection systems 102, an SEM or a 2132 inspection tool. The defect images tell a 

15 classification system 104 that defects have occurred, but does not classify the defects as to 
types or causes. After the defects are classified, as described below, they are sent to an 
analyzer 106, such as KLA-Tencor's KLArity system, an example of which is described in 
the above-referenced U.S. Application Serial Nos. 08/958,288 and 08/958,780. 

Fig. 1(b) is a block diagram showing an overview of a semiconductor optical, 

20 ebeam, or other types of inspection systems using the present invention for inspection set- 
up. As discussed above, defect images are preferably received from an outside source, such 
as an optical, ebeam, or other types of inspection systems 102, an SEM or a 2132 inspection 
tool. These images can relate to etch, photolithography, deposition, CMP, or to some other 
manufacturing process. The defect images tell a classification system 104 that defects have 

25 occurred, but does not classify the defects as to types or causes. After the defects are 
classified, as described below, they are sent to an analyzer 106, such as KLA-Tencor's 
KLArity system, an example of which is described in the above-referenced U.S. Application 
Serial Nos. 08/958,288 and 08/958,780. The output of the analyzer 106 is used as feedback 
to fine tune the inspection system. 

30 For example, depending on the number of errors found and the accuracy desired, the 

inspection system may be fine tuned to raise or lower the sensitivity of the system. For 
example, if the system is finding too many errors or errors that are not relevant to the 
particular manufacturing process, the system may be fine tuned to lower its sensitivity, 
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images according to common features. In a described embodiment, both the natural 
grouping 212 and the automatic classifier 204 use the same feature set. 

In addition, the human user can select images from the working set to be placed in a 
"training set" of images. The user then manually adds images/defects to the class/bins of 

5 the training set. Features are extracted from the selected images and stored along with the 
class/bin during a 'train classifier" operation. The classifier then classifies a set of images 
(such as the set W-T) and the user reviews the errors found in the classifier's decisions. For 
example, the user may view the confusion matrix to determine where the classifier differed 
form the user's classifications. The user then improves the training set by adding deleting, 

10 or reclassifying images via, e.g., a drag and drop interface and reassesses the classifier's 
performance until a satisfactory result is achieved. 

The images in the training set are sent to feature extractor software 206, which 
extracts a set of predefined features for each image in the training set. The data structures 
storing a set of features for an image is called the image's "feature vector." A feature vector 

15 contains the values for each feature for a particular image. 

The predefined features preferably extracted from the training set include, but are 
not limited to: 

a) features extracted from an image, such as: size, brightness, color, shape, texture, 
moment of inertia, context, proximity to wafer features or other defects, connectivity to 

20 adjacent features or other defects, other yield relevant properties derived from the image 
(e.g. short, open, bridging, particles, scratches, etc.) 

b) defect coordinates in wafers and spatial clusters of defect coordinates in the case 
of spatial cluster analysis, and 

c) other information pertaining to the defect that may be have been developed a 
25 priori, including but not limited to image type information such as in list a) and b), 

compositional or electrical information derived from analytic techniques and information 
pertaining to the processing history, yield relevance or origins of the defects in question. It 
will be understood that any appropriate features can be extracted without departing from the 
spirit of the present invention. Examples of analytical techniques used to derive 
30 compositional or electrical information are described in "Semiconductor Characterization: 
Present Status & Future Needs" ed. W.M. Bullis, D.G. Seiler, A.C. Diebold, American 
Institute of Physics 1996, ISBN 1-56396-503-8 which contains an overview of the myriad 
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• Better manual classification quality (repeatability) 

• Quicker, more effective classifier creation process 

The window of Fig. 3 includes a tool bar 302, which contains commands that allow 
5 the user to open and save classes and defect images, to sort images in the working set, and 
to create and manipulate the training set. A confidence setting area 304 allows the user to 
adjust the Confidence level, which is an adjustable setting of how close an unknown defect 
can be to a training set. Values preferably range from 0 (loosest setting) to 1 (tightest 
setting). Changes can be dynamically viewed in confusion matrix 306. 

10 Confusion matrix 306 is used to display the results of manual vs. automatic defect 

classification. A confusion matrix can be generated for both a current set of images or an 
explicitly selected set. The manual (human) classification results are displayed on the X- 
axis and the automatic classification by classifier 204 are displayed on the y-axis. Results 
of the comparison that are in agreement for all defect classes (where both manual and 

15 automatic classification results are in agreement) are displayed on the diagonal across the 
confusion matrix. 

An area 308 displays the working set of images. These images may be displayed in 
unsorted order or may by sorted or arranged by natural grouping, at the choice of the user. 
The user preferably drags and drops images from the working set gallery 308 into the 
20 training set gallery 3 1 0. Here, images in the training set are displayed arranged in user- 
specified classes. Training set area 312 displays the classes (also called "bins") that contain 
the composition of the training set, defines grouping, and allows the user to create new 
classes/bins. When this area 3 12 is active, the user can create new classes/bins using the 
toolbar 302. 

25 Natural grouping matrix 3 14 allows the user to view how the images in the working 

group are distributed in the natural groupings. The number of images in a group is 
represented by a number in an element of the matrix 314. The user can click on an element 
in the matrix 3 14 to view all defect images in a particular natural grouping. 

In summary, the user can optionally indicate (e.g., via the toolbar or a menu item) 

30 that he wants the working set images sorted in natural groupings. The user then drags and 
drops the images from area 308 into the classes/ bins of the training set 310/312 and 
indicates a "training" function. The training function stores the feature vectors of the user- 
selected images of the training set and stores them in connection with the classes/bins. 
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grouping method 606, which can be any appropriate method. In the described embodiment, 

the feature vectors of the images are grouped using a known Kohonen mapping technique. 

In the described embodiment, the Kohonen map is seeded with non-random numbers to 

improve stability of the grouping and to make the grouping repeatable. In some 
5 embodiments, the images are displayed in their natural groups (also called clusters), as 

shown in Fig. 6(b). In other embodiments, the images are arranged to reflect the actual 

Kohonnen maps layout. 

Certain embodiments use a Spatial Signature Analysis (SSA) Technique, as 

described in l)http:// www-ismv.ic.oml.gov/projects/SSA.html; 2)Gleason S.S., Tobin 
10 K.W., & Kamowski, T.P., "Spatial Signature Analysis of Semiconductor Defects for 

Manufacturing Problem Diagnosis", Solid State Technology, July, 1996; 

3) http://www.dym.com/ssa.htm; 

4) http://www.electroglas.eom/products/loiights_datasheets/spar_ds.h and 5) 
http://ww.oml.gov/Press_Releases/archive/n^l9980804-00.html, which are herein 

1 5 incorporated by reference. 

In addition, in certain embodiments, the analysis and classification are not limited to 
images, but can be performed on clusters themselves. In such an embodiment, the classifier 
receives "cluster-based features" instead of raw images as described in T.P. Kamowski, 
K.W. Tobin, S.S. Gleason, Fred Lakhani, SPIFs 24th Annual International Symposium on 

20 Metrology, Inspection and Process Control for Microlithography XIII, Santa Clara 
Convention Center, Santa Clara, CA, February, 1999, which is herein incorporated by 
reference. Such a system applies grouping and Kohonen mapping to clusters instead of to 
raw images. For non-image data, clustering is gathered by EDS specifiers (using an x-ray 
system for analysis in an e-beam system) or by SSA analysis. 

25 Fig. 7 shows an expanded view of training area 312. The user can drag images from 

the working group 308 into the displayed groups in area 310 or into the classes/bins of area 
312. In the displayed embodiment, each class/bin has a class code, a group code (reflecting 
its natural group) and a number of defect images currently assigned to the class/group. The 
user can, of course, add and delete classes/bins as he wishes (e.g., via the toolbar). 

30 If the user wants to add a new class/bin, the class/bin is added. Other wise, an 

existing class/bin is opened. The user then manually adds images/defects to the class. 
Features are extracted from the selected images and stored during a "train classifier" 
operation (e.g., via the toolbar). The classifier then classifies a set of images (such as the set 
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outputting the classes of the defect images being classified. Feature extractor 1058 receives 
images and outputs features of the images. 

Fig. 10(b) also shows an embodiment in which the classifier receives tool history 
1005 as an input. Tool history includes, for example, the maintenance history of the tools or 
5 machine performing the inspection process and/or the manufacturing process. If the tool 
has been maintained according to its suggested maintenance schedule, its data may be 
weighted more than data from an unmaintained tool. Tool History 1055 may also include a 
threshold of inspection value, indicating that maintenance must be found in order for the 
classifier to give credence to the data from that tool This threshold may vary for individual 

1 0 tools or may be the same for all the tools of a particular type or function. Tool history may 
also indicate, for example, whether two runs of semiconductors where taken from the same 
tool (or which tool they were taken from). Thus, tool history 1055 may include, for 
example, equipment Ids. If it is known, for example, that Tool A has had problems in the 
past, data from tool A may be treated differently than data from a trouble-free tool B. 

15 As described above, Fig. 10(c) shows that the inspection and analysis/classification 

process is performed in real-time during the inspection process instead of as a separate 
process. In such a system, inspection system 1052 is shown inside the system 1004 to 
indicate that it is part of the same system as the classifier 1056. The system utilizes review 
images ("patch" images) to effect real-time inspection. New inspection systems such as the 

20 KLA 2350 include ADC embedded inside the image computer running in f real time' during 
the inspection. This system, called an iADC (integrated ADC) system, works by grabbing 
patches' around the defect location during scan and performing ADC on the defective pixels 
in these patches. All this is done in hardware inside the inspector so that no addition 
throughput it required. 

25 Figs. 10(a), 10(b), and 10(c) each contain a dotted line 1003, 1053 depicting that, in 

certain systems, the classifier can provide feedback signals to the inspection system, in a 
similar manner discussed above in connection with Fig. 1 . 

It will be understood that various embodiments and alternations can exist without 
departing from the spirit and scope of the invention. For example, the concept of the 

30 invention can be extended to include automatically sorting images in the background (for 
example while running defect analysis software) by defect type, and then displaying the 
result as a wafer map with a distribution of each selected type shown over the wafer map. 
This is just one way of using the output data. Defect location distribution 
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What is Claimed is: 

1 . A method implemented by a data processing system for classifying a plurality of 
received images, comprising: 

5 extracting features from a training set that is a user-chosen subset of the 

plurality of images, each image in the training set having an associated class; 

classifying, by the data processing system, at least one of the plurality of 
images in accordance with the extracted features and classes of the training set; 

allowing a user to classify ones of the plurality of images; and 
10 displaying the results of a comparison between the classification by the data 

processing system and the classification by the user. 

2. The method of claim 1, wherein the features of the training set include size. 

15 3. The method of claim 1, wherein the features of the training set include 

brightness. 

4. The method of claim 1, wherein the features of the training set include color. 
20 5. The method of claim 1, wherein the features of the training set include shape. 

6. The method of claim 1, wherein the features of the training set consist at least one 
of: texture, moment of inertia, context, proximity to wafer features, proximity to other 
defects, connectivity to adjacent features, connectivity to other defects, and yield relevant 

25 properties derived from the image. 

7. The method of claim 1, wherein the features of the training set include defect 
coordinates in wafers. 

30 8. The method of claim 1, wherein the features of the training set include defect 

coordinates when spatial cluster analysis is used. 
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18. The method of claim 1, wherein the features include tool history information 
relating to the past success rate of the classification step. 

5 19. The method of claim 1, wherein only some of the plurality of images relate to an 

semiconductor etch process. 

20. A method implemented by a data processing system for classifying a plurality of 
received images, comprising: 

10 extracting features from a training set that is a user-chosen subset of the 

plurality of images, each image in the training set having an associated class; 

classifying, by the data processing system, at least one of the plurality of 
images in accordance with the extracted features and classes of the training set; 

displaying the results of a comparison between the classification by the data 
15 processing system and the classification by the user. 

21 . A system that classifies a plurality of received images, comprising: 

a software portion configured to extract features from a training set that is a 
user-chosen subset of the plurality of images, each image in the training set having an 
20 associated class; 

a software portion configured to classify, by the system, at least one of the 
plurality of images in accordance with the extracted features and classes of the training set; 

a software portion configured to allow a user to classify ones of the plurality 

of images; and 

25 a software portion configured to send feedback to an inspection system to 

fine-tune the inspection system in accordance with the user's classification. 

22. The system of claim 21, wherein the features of the training set include size. 

30 23. The system of claim 2 1 , wherein the features of the training set include 

brightness. 

24. The system of claim 21, wherein the features of the training set include color. 
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25. The system of claim 21 wherein r 

' herem featUres of *e training set include shape. 

26. The system of claim 21 wherein th* c a. 

* one of: texrore, ^ *— — «*, set cot* a, |M9 

27. The system of claim 21 wherein a «■ a 

10 coordinates in wafer,. ' ^ ^ ^ offee training set include defect 

28. The system of claim 21 wherein <w a. 

- when s« ia , . J. 6 ^ ° f *• "«* « delude defec, 



coordinates when spatial cluster analysis 



is used. 



15 



20 



29. The system of claim 21 wherein #k.« . 
defects. °m one of the processing history, yield, relevance, and origins of 

30. The system of claim 21 where 
*ep Mt yof imagKta ^^*7 e ^ OT ^^'o* s ifya, 1 ea s ,oneof 

technique. -"™"'««P"™.ly of images using a Kohonen mart 



31. The system of claim 30, wherein th-v u 

25 numbers. ^ *" K ° h ° nen ma P is with non-random 

32. The system of claim 21 where th~ ^ 
^P^ofiroageainaccordan^r.^ dassify a. ieas, one of 

- a portion ^^^'Tr ^ " *~ *** 

30 analysts technique. ""'''"^^^.Ktagaspaua, 

signature 
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33. The system of claim 21, where the portion configured to classify at least one of 
the plurality of images further includes a portion configured to classify in accordance with 
cluster based features instead of images. 



5 34. The system of claim 21 , wherein the portion configured to allow a user to 

classify ones of the plurality of images includes a portion configured to display the images 
to the user in classification groups determined by the classifying step. 

35. The system of claim 21, further comprising a portion configured to send 
10 feedback to an inspection system to fine tune the inspection system in accordance with the 
user's classification. 



36. The system of claim 21, further comprising: a portion configured to inspect an 
inspection object in real-time and to send the results of the inspection set to a classifier 

1 5 trained in accordance with the plurality of images classified by the user. 

37. The system of claim 21 , wherein the features include tool history information 
relating to an inspection system. 

20 38. The system of claim 21, wherein the features include tool history information 

relating to the past success rate of the classification step. 

39. The system of claim 21, wherein only some of the plurality of images relate to a 
semiconductor etch process. 

25 

40. A system that classifies a plurality of received images, comprising: 

a portion configured to extract features from a training set that is a user- 
chosen subset of the plurality of images, each image in the training set having an associated 
class; 

30 a portion configured to classify, by the system, at least one of the plurality of 

images in accordance with the extracted features and classes of the training set; 
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